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Abstract —Correlation between channel state and source sym¬ 
bol is under investigation for a joint source-channel coding 
problem. We investigate simultaneously the lossless transmission 
of information and the empirical coordination of channel inputs 
with the symbols of source and states. Empirical coordination is 
achievable if the sequences of source symbols, channel states, 
channel inputs and channel outputs are jointly typical for a 
target joint probability distribution. We characterize the joint 
distributions that are achievable under lossless decoding con¬ 
straint. The performance of the coordination is evaluated by 
an objective function. For example, we determine the minimal 
distortion between symbols of source and channel inputs for 
lossless decoding. We show that the correlation source/channel 
state improves the feasibility of the transmission. 

Index Terms —Shannon Theory, State-dependent Channels, 
Joint Source-Channel Coding Problem, Empirical Coordina¬ 
tion, Empirical Distribution of Symbols, Non-Causal Encod¬ 
ing/Decoding. 


L Introduction 

State-dependent channels with state information at the 
encoder have been widely studied since the publication of 
Gel’fand Pinsker’s article m. The authors characterize the 
channel capacity in the general case and Costa evaluates 
it for channels with additive white Gaussian noise in ||2|. 
Interestingly, additional noise does not lower the channel 
capacity while it is observed by the encoder. In this setting, the 
encoder has non-causal state information, i.e. it observes the 
whole sequence of channel states. This hypothesis is relevant 
for problems such as computer memory with defect El in 
which the encoder observes the sequence of defects or digital 
watermarking Q since the sequence of states is a watermarked 
image/data. Non-causal state information is also appropriate 
for the problem of empirical coordination 0, since the se¬ 
quence of source symbol acts as channel states. More recently, 
the notion of "state amplification" was introduced in 0. In 
this framework, the decoder is also interested in acquiring 
some information about the channel state. The amount of such 
information is measured by the "uncertainty reduction rate". 
The authors characterize the optimal trade-off region between 
reliable data rate and uncertainty reduction rate. As a special 
case, they consider that the encoder only transmit the sequence 
of channel-states. 


In this paper, we investigate the problem of correlation 
between channel state and source symbol with a constraint 
of empirical coordination. Although the problem of corre¬ 
lation state/source is different from the problem of state 
amplification, both problems coincide when considering the 
transmission of channel states only. The reliability criteria we 
consider is based on empirical coordination ||7], lH), ||9l, also 
referred as empirical correlation. An error occurs if sequences 
of source symbols, channel states, channel inputs and channel 
outputs are not jointly typical for a target joint probability, or if 
the decoding is not lossless. This result applies to coordination 
games 0, in which the encoder aims at maximizing an 
objective function that depends on symbols of source and 
channel while transmitting lossless information to the decoder. 



Fig. 1. Channel state S is correlated with information source U. Encoder 
C and decoder D implement a coding scheme such that sequences of 
symbols (U™, S", X”, E", O’") G AJ"(Q) are jointly typical for the target 
probability distribution Q{u,s,x,y,u) and the decoding is lossless. 


Channel model and definitions are presented in Sec. [Ill In 
Sec.Hni we characterize the set of achievable empirical distri¬ 
bution. In Sec. lYl we compare this result to previous works 
and to the case of non-correlated channel state/information 
source. SeclV] provides numerical results that illustrate the 
benefit of taking into account the correlation between channel 
state and information source. The conclusion is stated in Sec. 
[Vll and sketches of proof of the main result are presented in 
App. |A] and |B] 

II. System Model 

The problem under investigation is depicted in Fig. [D Cap¬ 
ital letter U denotes the random variable and lowercase letter 
u GU denotes the realization. We denote by t/". S'", X", Y", 
t/", the sequences of random variables of the source symbols 
u" = (ui,..., Un) € W", of channel state s" € S", of channel 
inputs x" G Y", of channel outputs y" G 3^" and of outputs 









of the decoder u" £ W". We assume the sets U, S, X, y are 
discrete and denotes the n-time cartesian product of set U. 
Notation A(^) stands for the set of probability distributions 
V{X) over X. The variational distance between probabilities 
Q and 7^is denoted by ||Q-7^||v = 

Notation l(M|rt) denotes the indicator function that equals 
1 if M = u and 0 otherwise. The Markov chain property is 
denoted hy Y -e-X -e-U and holds if for all (m, x, y) we have 
P(t/|x, u) = ^{y\x). The information source and channel states 
are i.i.d. distributed with V^s the channel is memoryless 
with transition probability 7^|xs- The statistics of V^s T^j^s 
are known by both encoder C and decoder V. 

Coding Process: A sequence of source symbols rt" £ W" 
and channel states s” £ 5" are drawn from the i.i.d. probabil¬ 
ity distribution defined by equation ([T]i. Encoder C observes 
m" £ W", s" £ <S" non-causally and sends a sequence of 
channel inputs x" £ TP". The sequence of channel outputs 
yU g yn jg according to the discrete and memoryless 

transition probability defined by equation (|2]l. 

n 

n 

r®"(z/”|x",s") = nT(t/z|x„s,)- (2) 

i=l 

The decoder observes the sequence of channel outputs j/" £ 
3^” and returns a sequence m" £ 77". We consider lossless 
decoding: the sequence of output of the decoder u" £ 77" 
should be equal to the source sequence m" = u" £ 77". 

The objective of this work is to characterize the set of 
empirical distributions Q £ A{h{ x S x X x y x 14) 
that are achievable, i.e. for which the decoding is lossless 
and the encoder and decoder can implement sequences of 
symbols ([/", 5”", X", F", [/") £ A*"(Q) that are jointly 
typical for probability distribution Q with high probability. 
The definition of typical sequence is stated pp. 27 in M- 
In this setting, the sequence of channel inputs X" must 
be coordinated {i.e. jointly typical) with (77", S'"). The per¬ 
formance of the coordination is evaluated by an objective 
function z/:77xSxA’x3^H>IR, as stated in m). For 
example, we characterize the minimal distortion level d{u, x) 
between the source symbol u € 14 and channel input x G X 
under lossless decoding constraint. Error probability evaluates 
simultaneously the lossless transmission of information and 
the empirical coordination of symbols of source, state, channel 
input, channel output and decoder output. Since the decoding 
is lossless, a joint distribution Q £ A(77 x5xA’x^x77)is 
achievable if the marginal probability distribution Q{u\u) = 
1(u|m) is equal to the indicator function. However, the lossless 
decoding condition is more restrictive than marginal condition 
Q(m|u) = 1(m|u), since sequences 77" = 77" should be 
strictly equal. 


a pair of functions c = (/, g) defined by equations @-(|4|l. 

/ : xS^ ^ X", (3) 

g: y'^^W. (4) 


The empirical distribution Q" G A{14 x S x X x y x 14) of 
sequences (u", s", x", y", m") £ 77" x 5" x X" x y" x 77" 
is defined by equation © where A/(u|w") denotes the number 
of occurrence of symbol u G 14 in sequence rt" £ 77". 


Q'^{u,s,x,y,u) 

V{u,s,x,y,u) 


G 


N{u, s, X, y, ulit", s", x", y", u") 
n 

U xS X X xy xU. (5) 


Fix a target probability distribution Q G A{14 x S x X x y x 
14), the error probability of the code c £ C(n) is defined by 
equation ®. 


( 6 ) 

where Q" £ A(77 x <S x X X y X 14) is the random variable 
of the empirical distribution of the sequences of symbols 
(77", S'", X", F", 77") £ 77" X S" X X" X y" X 77" induced 
by the code c G C(n) and probability distributions Vus, 7^|xs- 


Ve{c) = Vc 


g”-Q >s]+vju'^f 


Definition II.2 A probability distribution Q G A{14 x S x X x 
y X14) is achievable if for all e > 0, there exists a n £ N such 
that for all n > n there exists a code c G C(n) that satisfies: 

>e^+Vc 


77" 77 77" < e. (7) 


Ve{c) = Vc 


or-Q 


If the error probability Vq{c) is small, the empirical frequency 
of symbols {u,s,x,y,u) Gl4xSxXxyxl4 is close to 
the probability distribution Q{u, s, x, y, u), i.e. the sequence of 
symbols (77", S", X", F", 77") £ X*"(Q) are jointly typical 
for target distribution Q{u, s, x, y, u), with large probability. 
In that case, sequences of symbols are coordinated empirically. 


III. Channel State Correlated with the Source 

We consider probability distributions Vus for the source and 
7^|xs for the channel. We characterize the set of probability 
distribution Q(u, s, x, y, u) G A(77 x S x X xy xl4) that are 
achievable. 


Theorem III.l (Sonrce Correlated with Channel State) 

1) The joint probability distribution Q{u, s,x,y,u) is 
achievable if and only if it decomposes as follows: 

Q{u,s) = Vusiu,s), 

Q{y\x,s) = r(y|x,s), 

< Q{u\u) = 1(m|u), 

Y^{X,S)^U, 

Vus{u, s) (8) Q(x|m, s) (8) T(y|x, s) <8) 1 (u|m) is achievable. 

2) The probability distribution Vus(u, s) 0 Q(x|rt, s) 0 
T(y|x, s) 0 1(u|m) is achievable if: 

max ( 7(77, IF; F) - /(IF; S|77) - 7/(77) ) > 0, 

QeQ V / 


Definition II.l A code c G C(n) with non-causal encoder is 


( 8 ) 












3) The probability distribution 'Pus{u,s) ® Q(x\u,s) ® 
T{y\x, s) (8) l(u|w) is not achievable if: 

max ( I{U, W; V) - I(W; S'lt/) - H(U)) < 0, (9) 

SeQ V / 

where Q is the set of distributions QsA(WxiSxyVxA’x 
y X U) with auxiliary random variable W that satisfies: 

{ E™ew Qi'>^^s,x,w,y,u) 

= Pus(m, s) ® Q(x|m, s) ® T{y\x, s) 8 1(m|u), 

F ^ {X, S) ^ {U,W). 

Since Vusiu,s), Q{x\u,s), T{y\x,s), l('u|ii) are fixed, the 
set Q corresponds to the set of transition probability Qw|usx- 

Sketchs of proof of Theorem lIII.il are stated in Appendices 
|3and|B] We refer to equation (l8]l as "information constraint". 

Transition probability Q{x\u, s) is the unique degree 
of freedom for the joint distribution Q(u, s, x, y, u) since 
'Pus{u,s), T{y\x,s) and 1 (m|u) are the data of the problem. 
Q{x\u, s) characterizes the empirical coordination of the chan¬ 
nel inputs A” with sequences of source/state This 

result applies to optimization problems of objective functions 
v'. UxSxXxyv^M, that depend simultaneously on 
symbols {U, S, X,Y) of source and channel. 


Example III.2 (Minimal Distortion Source/Input Symbols) 

Consider distortion function d : IT x X for source/input 
symbols. The minimal distortion level D* is given by: 


D* 


min E 

Q.lusS-A* 


d{U,X) , 


( 10 ) 


where A* is the convex closure of the set of achievable Qx|u 5 -' 


A* = Co/iv|q ^|„5 i.I. max (/(!/, W; X) - /(W; S| t/) - ff (!/))> oj. (11) 

Equation (IlOb is a direct consequence of Theorem I///. 71 see 
also Ml?. Consider a smaller distortion level D° < D*. The 
corresponding transition probability Q°|jj 5 (f. A* is not achiev¬ 
able. There is no code c(n) such that X'^ is coordinated, i.e. 
jointly typical, with ((7", S'^)for target probability distribution 
Vus ® Hence, distortion level D° is not achievable. 

For any objective function v. Theorem IIII. 1 1 characterizes the 
coordination Qx|us G TX that is achievable and optimal ||5l : 

max E[v{U,S,X,Y)]. (12) 

CxIusGyt* 


Corollary III.3 (Side Information at the Decoder) 

Probability distribution Vusziu, s, z)®Q{x\u, s)®T[y\x, s)® 
l.{u\u) is achievable if: 

max(/((7,lF;y,Z)-/(VF;S'|(7)-77((7) ] >0, (13) 

SGQ V / 

where Q is the set of distributions Q & /S.{IA x S x Z xVV x 
X xy xlA) with auxiliary random variable W that satisfies: 

E™ew QW,s,z,x,w,y,u) 

= Vusziu, s, z) 8 Q{x\u, s) 8 T{y\x, s) 8 l(a|u), 

' Y^{X,S)^{U,Z,W), 

^{U,S) -e- {X,Y,W). 

Only achievability result is stated in Corollary IIII. 31 but the 
converse holds. Proofs are obtained by replacing the channel 
output Y by (Y, Z) in App. lAlandlBl 


Corollary III.4 (Removing Coordination of Channel Input) 

Lossless decoding is achievable if: 


max ( /([/, W; Y) - I{W] S'|(7) - H{U)] > 0. (14) 

Qxwlus V / 

Converse of Corollary IIII.4I also holds. As no coordination is 
required between X and {U, S), transition probability Qx|us 
is a degree of freedom for maximizing (fl4ll . Proof is a direct 
consequence of Theorem IIII. II since the maximum in (fT4t is 
taken over transitions Qx|us and Qw|usx instead of Qw|usx only. 


5 ' = 0 


S'= 1 


a: y 


Po 0 ^ ^-7^ 0 



2 


Fig. 3. Channel T(y|x, s) with states S € {0,1} depending on error 
probability £ £ [0,0.5]. Probabilities (po?PiiP 2 ,P 3 ,P 4 ,?> 5 ) characterize 
the transition probability Q(ai|s). The unique difference between states 
S € (0,1} holds for the channel input X = 2. Exploitation of the con'elation 
between channel state S and source symbol U improves the transmission. 
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Fig. 2. Decoder observes a side information Z correlated with {U, S). 


IV. Comparison with Previous Work 

In this section, we consider the result stated in Corollary 
IIII.4l for lossless decoding, removing the coordination between 
X and {U,S). We compare our result to the case of "State 
Amplification" ||6l and independent source/state {U,S), IfTTI . 


Consider decoder observes a side information Z correlated 
with {U,S), drawn i.i.d. with V^sz, as stated in Fig. |2l 


Corollary IV.l (Comparison with Previous Work) 

• When the source is the state S = U, information constraint 


































dl reduces to eq. (2) in ^ for "state amplification": 


maxI{S,X-,Y)- H{S) >0. (15) 

Sxls 

• When s ourc e U is independent of state S, ii^rmation 
constraint dl reduces to the one stated in SB, fB: 

™®^Sxw|us > 0 

<l=> (^I(W; Y) - I(W; S) - H{U)^ > 0. (16) 

Proof of equation ( fTSI ) in Corollary IIV.II is stated in IfT^ . As 
mentioned in SB, separation holds when random variables of 
the channel {S,X,Y) are independent of the source U. 

Proposition IV.2 The benefit of exploiting the correlation 
between source U and channel state S is greater than: 

max (^I{U, W; Y) - I{W; 5|i7) - H{U)^ 

- max (l{W;Y) - I{W;S) - H(U)) >mmI{U;Y). (17) 

Sx„|s V y Q,|3 

Proof of Proposition I1V.2I is stated in ifT^ . 

V. Numerical Results without Coordination 
We consider the channel with state T{y\x,s) depicted in 
figure [3 depending on parameter e € [0, 0.5] and the joint 
probability distribution Vusiuis) defined by figure H The 
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Fig. 5. Upper bound GD on GD is smaller than the lower bound GD on 
information constraint GD of Corollary liiOl 

consider the maximization over parameters (P 3 ,P 4 ,P 5 ). 

• For a = 0.99 in Fig. 0 the upper bound (GSl l on ( fT6b is 
always smaller than the lower bound (G9] l on our information 
constraint d- For some values of e, the decoder can recover 
the source. Exploitation of the correlation between channel 
state S and source symbol U improves the transmission. 


S' = 0 5 = 1 

(7 = 0 

U = 1 


l + CH 

4 

1 —a 

4 

1 —a 

4 

l+cn 

4 


• For £ = 0.101 in Fig. |6] upper bound (GSl) on (O 
is negative and does not depend on correlation parameter 
a. For large values of a, the (G^ lower bound on (G^ 
is positive. Hence, the decoder can recover the information 
source exploiting the correlation between U and S. 


Eig. 4. Joint probability distribution Pus(u, s). If o = 0, the random 
variables U and S are independent. If a = 1, both random variables are 
equal: U = S. The marginal probability distributions on U and S are always 
equal to the uniform probability distribution (0.5, 0.5). 

correlation between the information source U and the channel 
state S depends on parameter a € [0,1]. We evaluate the 
benefit of exploiting the correlation between source U and 
channel state S for different values of parameters a G [0,1] 
and £ G [0,0.5]. Since the information constraint with auxil¬ 
iary random variable W is difficult to characterize, we consider 
the following upper (GSl l and lower (G9] l bounds: 

GD < max (l{X-,Y\S)-H{U)), (18) 

(P3,P4.P5) v y 

G3 > max (l{X-,Y\U)-I{X-,S\U)-H{U\Y)). (19) 

(P3,P4.P5) v y 

• (GSl l is an upper bound on (Ghl) that does not take into 
account the correlation between U and S. 

• ( G9l l is a lower bound on (GGl of Corollary 1111.41 that takes 
into account the correlation between U and S. 

Only parameters {p 3 ,Pi,P 5 ) are chosen in order to maximize 
the lower bound (G9l l on the information constraint. Regarding 
upper bound (GST l. since parameters {po,pi,P 2 ) = (jj 3 ) 
maximize the mutual information I{X-,Y\S = 0), we only 



Fig. 6. Upper bound GD on GB does not depend on a. since the correlation 
between U and S is not taken into account. 

VI. Conclusion 

We consider a state dependent channel in which the channel 
state is correlated with the source symbol. We investigate 
simultaneously the lossless transmission of information and 
the empirical coordination of channel inputs with the symbols 









































of source and states. We characterize the set of achievable 
empirical distributions with or without side information at 
the decoder. Based on this result, we determine the minimal 
distortion between symbols of source and channel inputs under 
lossless decoding constraint. Correlation between information 
source and channel state improves the feasibility of the lossless 
transmission. 


Appendix A 

Sketch of Achievability of Theorem IIII.1I 
There exists ^ > 0, rates Rm > 0 and Rl > 0 such that : 



Rm 

> 

H{U)y5, 

(20) 


Rl 

> 

I{W-,S\U)y5, 

(21) 


Rivi + Rl 

< 

I{U,W]Y)-5. 

(22) 

Random 

codebook. 

Generate \AA\ = 2"*^“ 

sequence 

(7"(m) 

with index 

m 

G A4 drawn from 

the i.i.d. 


marginale probability distribution For each 

index m G A4, generate \M.\_\ = 2"’^'- sequences 
G W" with index I G A4l, drawn from the 
i.i.d. marginal probability distribution Q®|" depending 
on U^{m). 


• Encoding function. The encoder observes the sequence 
of source symbols C/" G W" and finds the index m G Ad 
such that [/" = u^(m). It finds the index I G A4 l such 
that the sequence w'^(m,l) G A*^{U'^,S^) is jointly 
typical with the source sequences Encoder 

sends the sequence X" drawn from the probability 
2^wus depending on sequences (w”(m, Z), S'"). 


• Decoding function. The decoder observes the sequence 
F” and finds the indexes {m,l) G AA x A4 l such that 
(u"(to),/)) G A*"(F"). It returns sequence 
ti" = u^^m). 


From Packing and Covering Lemmas stated in OOl pp. 
46 and 208, equations (l20l l, (1211 1. (l22l l imply the expected 
probability of error events are bounded by e for all n>n: 


•Ec 


Vm e M, [7"^( 


< e. 


(23) 

Ec 

[e( 

Fi e aIl, 

>, 1) g A" 

Gu"(m), S") 


(24) 

Ec 

[e( 

(m, 

1), S.t. (^"(7 


')) e A*"(Y") 

)] < e,(25) 

Ec 


3m' ^ m, s.t. (f/ 

"(TT.'), W" 

(m'.l)) e Af 

‘(V”))l < e. 

(26) 


There exists a code c* G C(n) with Ve{c*) < Ser for 
all n > n. Lossless decoding condition is satisfied since 
f U^) < 4s. Sequences ,U^) G 

are jointly typical with high probability, hence em¬ 
pirical coordination requirement is satisfied. 


Appendix B 

Sketch of Converse of Theorem IIII.1I 

Define error event L; = 1 if ([/", S'", AT", F", 17") i AJ" 
and E = 0 otherwise. 


n ■ H{U) 

= H(U") 

< = 0) + 71 ■ e 

= r(a"; V”|E = 0) + E = 0) + n ■ e 

< /(E"; y”|E = 0) + ti ■ 2e 

= 53 r(E"; , E = 0) + n ■ 2e 

i=l 

< Fj, E = 0) + n ■ 2e 
i=l 

= ^ = 0 ) 
i=l 

- E = 0) + n ■ 2e 

i=l 

i=l 

Y^~^IU'^, E = 0) + 71 ■ 2e 

i=l 

= Y‘^~^ , |E = 0) 

i=l 

Y‘^~^, E = 0) + n ■ 3e 

i=l 

n n 

= = 0) - H E = 0) + n • 3e 

i=l i=l 

< n ■ max (l(U, W; Y) - I(W; S|E) ) + n. • 4e. 

QeQ ^ 


(27) 

(28) 

(29) 

(30) 

(31) 

(32) 


(33) 


(34) 


(35) 

(36) 

(37) 


Eq. dZTl t comes from the i.i.d. property of the source. 

Eq. (|28] > is due to Eano for coordination; V(E — 1) < e, ifT^ . 
Eq. (l30l t is due to Fano’s inequality for lossless decoding f[2\ . 
Eq. (|29l t, (ISTl t, and (l33T t come from properties of ML 
Eq. (l34l t comes from the Csiszar sum identity. 

Eq. (l35] t is due to {Ui,Si) independent of ifT^ . 

Eq. (|3^ is due to auxiliary RV, Wi = ([/“*, ■ 

Eq. (123 is due to continuity of MI and maximum over 
auxiliary random variables W s.t. F -e- {X, S) -e- IF, ifT^ . 
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